The present invention relates to data processing and more particularly to data preprocessing.
In U.S. Pat. No. 8,218,555, U.S. Patent Application Publication No. 2010/0241758, U.S. Pat. No. 8,131,880, U.S. Pat. No. 8,069,102, U.S. Pat. No. 7,284,070, U.S. Patent Application Publication No. 2012/0089497, U.S. Patent Application Publication No. 2007/0255866, U.S. Pat. No. 8,130,758, and U.S. Patent Application Publication No. 2012/0089694, various preprocessing units are disclosed. These preprocessing units preprocess data packets received from sender computers in a network and then transfer application data (i.e., data destined for an application, preferably a software application, running on a receiver computer) contained in the data packets to the receiver computer or other components of the receiver computer if the preprocessing unit is a component of the receiver computer, thereby reducing the load on the receiver computer's memory, bus, and CPU. The transferred application data may be in the original format or a different format after decoding. The preprocessing may include the decoding of multiple network protocols, the stripping of header information, the filtering of data packets in accordance with time predicates or parameters established by a user, performing data reduction operations such as, for example, the calculation of specific prices in the case of application data which is financial data. The preprocessing units may include processors or configurable circuits and may even generate and send a response to the sender computer of a data packet.
U.S. Patent Application Publication No. 2010/0241758 discloses a preprocessing unit including a data receiver to receive a data packet containing packet information and application data, a relevance checker adapted to determine the relevance of said data packet in dependence on said packet information, an output circuit adapted to output preprocessor output data, and a first controller adapted to control the output of preprocessor output data in dependence on the relevance of said data packet. In this context, preprocessor output data are output data of the processor, packet information is data containing information about said data packet, and application data information is data containing information of the application data.
In multicast delivery methods, a single sender computer sends data packets to a group of receiver computers. An advantage of these delivery methods is that they cause relatively low traffic if the data packets comprise application data to be used by several of the receiver computers. Common multicast delivery methods are based on the user datagram protocol (UDP), have no handshaking dialogues (no confirmation is sent from the receiver computers to the sender computer that a data packets has been received), and are therefore prone to data loss. In order to increase the reliability of these and similar delivery methods, identical application data are sent redundantly (at least twice) from one or several sender computers to one or several receiver computers. Disadvantageously, the preprocessing unit disclosed in U.S. Patent Application Publication No. 2010/0241758 is not able to discard redundant application data, resulting in a considerably increased load on the memory, bus, and CPU of the receiver computer.